﻿{
 "cells": [
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "# KNN\n",
    "\n",
    "---\n",
    "\n",
    "## 🧠 一、KNN 是什么？\n",
    "\n",
    "KNN 是一种**基于实例的学习方法（Instance-based Learning）**，它不对训练数据进行显式的建模，而是保留全部数据，在预测时使用它们来推断新样本的类别或值，因此也被称为**懒惰学习算法（Lazy Learning）**。\n",
    "\n",
    "---\n",
    "\n",
    "## 🧩 二、KNN 分类算法原理详解\n",
    "\n",
    "### 1️⃣ 步骤一：选择K值\n",
    "\n",
    "- K 是一个正整数，表示要参考的“最近邻”的个数。\n",
    "- K 值对结果有很大影响：\n",
    "  - K太小：容易受噪声干扰，模型复杂，容易过拟合。\n",
    "  - K太大：邻居太多，容易欠拟合，降低模型的区分能力。\n",
    "\n",
    "👉 一般使用**交叉验证**或经验法则来选择K值。\n",
    "\n",
    "---\n",
    "\n",
    "### 2️⃣ 步骤二：计算距离\n",
    "\n",
    "常见距离度量方法：\n",
    "\n",
    "#### 🔵 欧几里得距离（最常用）：\n",
    "\n",
    "对于两个样本 $x = (x_1, x_2, ..., x_n)$ 和 $y = (y_1, y_2, ..., y_n)$，欧几里得距离为：\n",
    "\n",
    "$$\n",
    "d(x, y) = \\sqrt{ \\sum_{i=1}^n (x_i - y_i)^2 }\n",
    "$$\n",
    "\n",
    "#### 🔵 曼哈顿距离：\n",
    "\n",
    "$$\n",
    "d(x, y) = \\sum_{i=1}^n |x_i - y_i|\n",
    "$$\n",
    "\n",
    "#### 🔵 闵可夫斯基距离（更通用）：\n",
    "\n",
    "$$\n",
    "d(x, y) = \\left( \\sum_{i=1}^n |x_i - y_i|^p \\right)^{1/p}\n",
    "$$\n",
    "\n",
    "scikit-learn 默认使用欧几里得距离（`p=2`）。\n",
    "\n",
    "---\n",
    "\n",
    "### 3️⃣ 步骤三：选出最近的K个点\n",
    "\n",
    "- 将训练集中所有样本与测试样本计算距离\n",
    "- 排序后选择距离最小的K个训练样本\n",
    "\n",
    "---\n",
    "\n",
    "### 4️⃣ 步骤四：进行“投票”或“平均”\n",
    "\n",
    "#### ✅ 分类：\n",
    "\n",
    "- **多数投票法（Majority Voting）**：K个邻居中哪个类别最多，就预测为哪个类别\n",
    "- 可以加入**权重**：例如按距离加权，离得近的邻居投票权重更大（`weights='distance'`）\n",
    "\n",
    "#### 📏 回归：\n",
    "\n",
    "- 预测值为K个邻居目标值的**平均值**（或加权平均）\n",
    "\n",
    "---\n",
    "\n",
    "## 🧬 三、KNN 的一些关键点\n",
    "\n",
    "### 🔧 1. 特征归一化（标准化）\n",
    "\n",
    "因为 KNN 依赖距离，如果特征单位差别太大，会导致某些特征主导距离计算。例如“身高（cm）”和“收入（万元）”。\n",
    "\n",
    "常用方法：\n",
    "- **标准化（Z-Score）**：让特征均值为0，标准差为1\n",
    "- **归一化（Min-Max）**：把特征压缩到[0, 1]\n",
    "\n",
    "```python\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler = StandardScaler()\n",
    "X_scaled = scaler.fit_transform(X)\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "### 🚀 2. 适用场景和优缺点\n",
    "\n",
    "#### ✅ 优点：\n",
    "- 简单易实现\n",
    "- 无需训练过程，适合快速建模\n",
    "- 对多分类问题效果好\n",
    "\n",
    "#### ❌ 缺点：\n",
    "- 预测慢（每次都要全量计算距离）\n",
    "- 对维度敏感（“维度灾难”）\n",
    "- 对噪声和离群点敏感\n",
    "\n",
    "---\n",
    "\n",
    "## 📌 四、KNN小总结\n",
    "\n",
    "| 特性   | 内容             |\n",
    "|------|----------------|\n",
    "| 类型   | 监督学习，懒惰学习      |\n",
    "| 用途   | 分类、回归          |\n",
    "| 关键参数 | K值，距离函数，权重方式   |\n",
    "| 要点   | 特征归一化，K选择，计算效率 |"
   ],
   "id": "78725a562601baa5"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-04-30T15:54:17.858581Z",
     "start_time": "2025-04-30T15:54:16.835303Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from sklearn.datasets import load_iris\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "# 1. 加载数据\n",
    "iris = load_iris()\n",
    "X = iris.data       # 特征：花萼长宽、花瓣长宽\n",
    "y = iris.target     # 标签：三种鸢尾花类别\n",
    "\n",
    "# 2. 划分训练集与测试集\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)\n",
    "\n",
    "# 3. 创建KNN模型，设K=3\n",
    "knn = KNeighborsClassifier(n_neighbors=3)\n",
    "\n",
    "# 4. 训练模型\n",
    "knn.fit(X_train, y_train)\n",
    "\n",
    "# 5. 预测测试集\n",
    "y_pred = knn.predict(X_test)\n",
    "\n",
    "# 6. 评估准确率\n",
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "print(f\"KNN分类准确率: {accuracy:.2f}\")\n",
    "\n",
    "for i in range(5):\n",
    "    print(f\"样本{i} - 真实标签: {y_test[i]}，预测标签: {y_pred[i]}\")\n"
   ],
   "id": "2192a1b8df8b4ac",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KNN分类准确率: 1.00\n",
      "样本0 - 真实标签: 1，预测标签: 1\n",
      "样本1 - 真实标签: 0，预测标签: 0\n",
      "样本2 - 真实标签: 2，预测标签: 2\n",
      "样本3 - 真实标签: 1，预测标签: 1\n",
      "样本4 - 真实标签: 1，预测标签: 1\n"
     ]
    }
   ],
   "execution_count": 1
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
